avoid relatedTemplate ever returning ""

add: Fix crash adding filenames that are exactly 21 bytes long and begin
with a utf-8 character.

Also longer filenames that start with "....." would cause the same crash.

I also audited for other calls to truncateFilePath that could truncate it
to "". Most use pathmax so are not a problem. Backend.Utilities.genKeyName
could possibly truncate it like that, but appends the md5 so would not be a
problem either.

Sponsored-by: Kevin Mueller

diff --git a/CHANGELOG b/CHANGELOG
index dfd780321a7f9e193743f1d705f387d453150f24..978e86ec552345628a9dedfa6347d5b981f5f5fd 100644 (file)
--- a/CHANGELOG
+++ b/CHANGELOG
@@ -1,6 +1,8 @@
 git-annex (10.20250829) UNRELEASED; urgency=medium
 
   * drop: --fast support when dropping from a remote.
+  * add: Fix crash adding filenames that are exactly 21 bytes long and
+    begin with a utf-8 character.
 
  -- Joey Hess <id@joeyh.name>  Fri, 29 Aug 2025 12:34:06 -0400
 

diff --git a/Utility/FileSystemEncoding.hs b/Utility/FileSystemEncoding.hs
index d66d8a008cca7a45233d38c886cd75c4f9d70117..2fb726f1fc6714a6247fdea450e52fb4d9da35db 100644 (file)
--- a/Utility/FileSystemEncoding.hs
+++ b/Utility/FileSystemEncoding.hs
@@ -119,11 +119,14 @@ fromRawFilePath = decodeBS
 toRawFilePath :: FilePath -> RawFilePath
 toRawFilePath = encodeBS
 
-{- Truncates a FilePath to the given number of bytes (or less),
+{- Truncates a path to the given number of bytes (or less),
  - as represented on disk.
  -
  - Avoids returning an invalid part of a unicode byte sequence, at the
  - cost of efficiency when running on a large FilePath.
+ -
+ - Note that this may return ""! That can happen if it is asked to truncate
+ - to eg 1 byte, but the input path starts with a unicode byte sequence.
  -}
 truncateFilePath :: Int -> RawFilePath -> RawFilePath
 #ifndef mingw32_HOST_OS

diff --git a/Utility/Tmp.hs b/Utility/Tmp.hs
index f373ca6c1c8cefc2ab31fbb7f4917ddcfbcd3e63..c47cdfcb0bdabf938fa2ff4e80b7d97e106306c4 100644 (file)
--- a/Utility/Tmp.hs
+++ b/Utility/Tmp.hs
@@ -120,8 +120,11 @@ relatedTemplate' f
                 - ending in ".", and others like VFAT don't allow a
                 - filename to end with trailing whitespace, so avoid
                 - truncating a filename to end that way. -}
-               B.dropWhileEnd disallowed $
+               let p = B.dropWhileEnd disallowed $
                        truncateFilePath (len - templateAddedLength) f
+               in if B.null p
+                       then "t"
+                       else p
        | otherwise = f
   where
        len = B.length f

diff --git a/doc/bugs/git-annex_add__47__unlock_fails_for_some_names.mdwn b/doc/bugs/git-annex_add__47__unlock_fails_for_some_names.mdwn
index 0dce8ad00cabc8cf33dacfdfac53f417cb1b0c2c..59823cfbbd25e16c09f909a520340ad250f0f815 100644 (file)
--- a/doc/bugs/git-annex_add__47__unlock_fails_for_some_names.mdwn
+++ b/doc/bugs/git-annex_add__47__unlock_fails_for_some_names.mdwn
@@ -93,3 +93,5 @@ Yes, git-annex has been fantastic for managing large datasets across multiple ma
 ---
 
 This issue appears to affect **all Cyrillic filenames**, not just the initially identified patterns, making the current version of git-annex barely usable for repositories containing non-Latin filenames.
+
+> [[fixed|done]] --[[Joey]]

diff --git a/doc/bugs/git-annex_add__47__unlock_fails_for_some_names/comment_1_af33dfae3ccbc24f84c84612337b98bc._comment b/doc/bugs/git-annex_add__47__unlock_fails_for_some_names/comment_1_af33dfae3ccbc24f84c84612337b98bc._comment
new file mode 100644 (file)
index 0000000..cd36ba1
--- /dev/null
+++ b/doc/bugs/git-annex_add__47__unlock_fails_for_some_names/comment_1_af33dfae3ccbc24f84c84612337b98bc._comment
@@ -0,0 +1,27 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 1"""
+ date="2025-09-04T16:00:49Z"
+ content="""
+Reproduced. Thank you for an excellent bug report.
+
+And it is the temp filename generation causing the problem.
+
+       mkdir(".git/annex/othertmp/.0", 0777)   = 0
+       unlink(".git/annex/othertmp/.0")        = -1 EISDIR (Is a directory)
+       symlink(".git/annex/objects/k8/wf/SHA256E-s3--98ea6e4f216f2fb4b69fff9b3a44842c38686ca685f3f55dc48c5d3fb1107be4.md/SHA256E-s3--98ea6e4f216f2fb4b69fff9b3a44842c38686ca685f3f55dc48c5d3fb1107be4.md", ".git/annex/othertmp/.0") = -1 EEXIST (File exists)
+
+The cause is that relatedTemplate is returning "", which is not something the code
+is prepared for. That results in the ".0" directory name, and `".0" </> "" == ".0"`
+so it uses the same path for the temp file as for the subdirectory.
+
+Not all cyrllic names are affected though. Only ones that are exactly 21
+bytes long. Longer or shorter are both ok.
+
+The reason is that relatedTemplate wants to reserve 20 bytes for the random
+part of the temp filename. With a 21 byte filename, that means it wants to
+truncate it to 1 byte. But it that lands in the middle of the first unicode
+character, which is not allowed, so it truncates it to 0 bytes instead.
+
+I've fixed this bug.
+"""]]